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Abstract 

Despite 40 years of research into evidence-based policy (EBP) and a continued drive from both policymakers and 
researchers to increase research uptake in policy, barriers to the use of evidence are persistently identified in the 
literature. However, it is not clear what explains this persistence - whether they represent real factors, or if they are 
artefacts of approaches used to study EBP. Based on an updated review, this paper analyses this literature to explain 
persistent barriers and facilitators. We critically describe the literature in terms of its theoretical underpinnings, 
definitions of 'evidence', methods, and underlying assumptions of research in the field, and aim to illuminate the 
EBP discourse by comparison with approaches from other fields. Much of the research in this area is theoretically 
naive, focusing primarily on the uptake of research evidence as opposed to evidence defined more broadly, and 
privileging academics' research priorities over those of policymakers. Little empirical data analysing the processes or 
impact of evidence use in policy is available to inform researchers or decision-makers. EBP research often assumes 
that policymakers do not use evidence and that more evidence - meaning research evidence - use would benefit 
policymakers and populations. We argue that these assumptions are unsupported, biasing much of EBP research. 
The agenda of 'getting evidence into policy' has side-lined the empirical description and analysis of how research and 
policy actually interact in vivo. Rather than asking how research evidence can be made more influential, academics 
should aim to understand what influences and constitutes policy, and produce more critically and theoretically 
informed studies of decision-making. We question the main assumptions made by EBP researchers, explore the 
implications of doing so, and propose new directions for EBP research, and health policy. 
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Introduction: the evidence-based policy 
movement 

Although sceptics can be found, few researchers or 
policymakers would publicly disagree that evidence-based 
policy (EBP) is a goal for both academe and government. 
Reports describing the importance - and difficulty of - 
achieving this goal are published with regularity [1-3]. 
Perhaps unlike other disciplines, EBP has staunch 
advocates who contribute to an ever-increasing body 
of commentary around the subject. 

EBP is sometimes said to have derived from evidence- 
based medicine (EBM), which dates back at least to 1972, 
with Archie Cochrane's seminal work on effectiveness and 
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efficiency [4]. Since the early 1970s, both practitioners and 
academics have also considered how policy - in the 
sense of larger-scale decisions about the delivery and 
management of services at a population level - could 
be based on, or informed by, evidence [5,6]. For 
Cochrane and his heirs, the goal of EBM was to bring 
about the abandonment of harmful and ineffective 
interventions, and the adoption of interventions shown to 
be effective for clinical outcomes. This was to be achieved 
by implementing findings from systematic reviews and 
meta-analyses of robust outcome evaluations, ideally 
randomised trials [7-10]. 

However, this straightforward narrative of evaluation- 
based EBP modelled on medicine has long existed in a 
broader landscape of initiatives to foster closer and more 
effective links between research and policy (or between 
researchers and policy- makers). In the UK, for example, 
these ideas can be traced back at least to the Rothschild 
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experiment, evaluated by Kogan and Henkel in 1983 
[11]. The aim of this funding initiative was to enable the 
health research system to respond to policymakers' 
priorities for research [12]. The experiment was 
largely abandoned, which Kogan and Henkel ascribed to 
cultural differences between researchers and policymakers, 
the need for interaction, and other barriers [12,13]. Recent 
publications, again in the UK, from the Government Office 
for Science have provided guides for both academics and 
policymakers who wish to engage with the other group 
[14,15], and there is increasing interest within academic 
and higher education bodies generally about how academics 
may be able to increase their impact, the nature of this im- 
pact, and implications of the 'impact agenda' [16-18]. 
However, the tenor of these publications still tends to 
focus on promoting the use of academic research, rather 
than studying the practices of knowledge production and 
policymaking and implementation. 

As a result, specific questions about the use of research 
within policy have become divorced from a broader 
perspective on policy-making. Many researchers in what 
may be termed 'applied research' fields use terms like 
'knowledge translation', 'knowledge exchange', or 'evidence 
use' without providing clear definitions of what knowledge 
is part of which decision-making process [19-21]. The 
result has been a loss of clarity with respect to how 
evidence is supposed to improve decision-making, what 
constitutes and defines 'evidence' and 'policy', and which 
processes and outcomes are targeted by political or 
academic efforts. Nevertheless, both EBM and EBP have 
achieved substantial financial and political support, and 
substantial weight by the creation of a collection of 
organisations dedicated to producing evidence-based 
policy and practice recommendations, guidelines, and best 
practice statements, such as the What Works centres [22]. 
For health policy and health management in particular, 
there remains a prominent discourse about moral, ethical, 
political, and often financial imperatives to use evidence 
to make the best (value) decisions [23] - without, ideally, 
disenfranchising non-expert publics [24] . The study of the 
use of evidence in policy varies from negative to positive 
advocacy, from simplistic to complex understandings 
of the processes involved, from uncritical technical 
approaches to highly cynical commentary. Broadly, 
practitioners and academics have focused on facets of 
the use of evidence by policymakers and practitioners, 
have written polemics encouraging colleagues to do so 
[23,25-29], identified barriers and facilitators into evidence 
use [30,31], and designed interventions to increase the use 
of evidence by policymakers [32]. Evidence-based policy 
and practice, knowledge translation, and related concepts 
have become touchstones across a vast range of disciplines 
- almost sub-disciplines in their own right, with canons 
and conceptual toolkits of their own. 



While much of this work remains mainly theoretical 
(e.g., [33]), there is a rapidly growing empirical evidence 
base on barriers and facilitators of evidence use. Some 
idea of the extent and nature of this literature can be 
gained by looking at systematic reviews. Three reviews 
are particularly relevant here [30,31,34]; their methods, 
findings, and conclusions are summarised in Table 1. 

Common findings across all three include the im- 
portance of personal relationships and contacts be- 
tween decision-makers and researchers, and the need 
for research to be clearly and accessibly presented. Cul- 
tural and practical barriers to the use of evidence by 
policy-makers are identified. Finally, all three make the 
point that policymakers' definitions of evidence do not 
match academic constructions of 'evidence'. All three 
reviews also point to gaps in the literature as priorities for 
further research, but differ in their identification of these 
gaps. Orton focuses on the need for evaluation re- 
search of strategies to increase the uptake of research 
evidence [31], while Innvaer et al. [34] and Oliver et al. 
[30] are both more circumspect, pointing out that despite 
the size of the evidence base, much about policy-makers' 
attitudes to research evidence remains unclear. Innvaer 
et al. show how the limited available evidence mainly 
describes policymakers' beliefs and attitudes, rather 
than actual behaviours, and hence cannot be used as a 
basis to make strong recommendations [34]. Perhaps 
more importantly, both reviews note that there are few 
grounds by which to make firm recommendations or 
conclusions about the process, impact, or effectiveness 
of research in policy. Only rarely is enough detail 
known about the policy process to be able to com- 
ment usefully: for example, who are the main actors, 
where are decisions made, and how evidence fits into 
the process. 

The structure of this paper is as follows. First, drawing 
on a dataset from a recent systematic review of barriers 
and facilitators [30], we offer a high-level overview of 
the literature on evidence use in health policy, drawing 
out broad theoretical and definitional commitments. 
This paper does not primarily consider the findings of 
those studies, which are summarised above and set out 
in more detail elsewhere [30]. We also draw on similar 
findings about policymakers in non-health fields [35]. 
We set these data in the context of the wider theoretical 
literature about evidence and policy, drawing insights 
from policy sciences. Secondly, we aim to identify and 
challenge some of the more normative assumptions 
which are widely prevalent (if often implicit) in the 
EBP literature, particularly the following: that the 
policy-evidence 'gap' needs 'bridging'; that policy is 
usually not based on any data; that policy requires research 
evidence, preferably evaluative intervention research 
evidence; and that if more evidence were used in 
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Table 1 Comparing three systematic reviews on evidence-based policy 



Orton, 201 1 



Innvaar, 2002 



Oliver, 2014 



Aims 



nclusion criteria 
and search date 



Types of studies 
included 



To synthesise evidence about the 
extent, types, process of evidence 
use, and barriers and facilitators 

Europe, Canada, Australia, and NZ 
only, must explore "how research 
evidence is used in decision-making 
for public health". Primary studies only 

18 included, interviews and surveys 



Types of results PMs perceptions about the use 



presented 

Use of 
evidence? 



Synthesis and 
QA used 

Main facilitators 



Main barriers 



Theory 

Conclusions and 
implications 



of evidence; qualitative and 
closed-response 

Self-reported use of evidence in 
2 studies, undermined by quality 
concerns 

Narrative synthesis; CASP-based QA 

Improved relationships; researchers 
trained to disseminate, clear, relevant 
and easy-access research; PM trained 
in research skills; change of policy culture 

Unclear, irrelevant or low-quality 
evidence. 'Gulf between researchers 
and policymakers. Lack of PM research 
skills. Other pressures; practical constraints: 
financial, time frames, access to research, 
presentation, and interpretation 

None cited 

Action to address the barriers and 
facilitators needs to be taken; training 
to overcome barriers to research use; 
research on interventions to increase 
research uptake 



Synthesise facilitators of and barriers 
to the use of research evidence by 
health policymakers 

Interview studies with health policy 
decision-makers responsible for 
decisions on behalf of organisations 

24 studies in 26 papers, interviews 
and surveys 

PMs perceptions about the use of 
evidence; qualitative and closed-response. 
Types of 'use', theories about evidence use 

21/24 examined actual decision-making 
processes, all measured perceptions of use 
or hypothetical use of evidence 



To update Innvaer 2002 and identify new 
evidence in this area 



All studies reporting barriers or facilitators 
of use of evidence, from 2002-201 1 



145 included studies: 13 systematic reviews, 
42 interviews/qualitative studies, 13/25 
entirely/included survey 

Perceived and observed factors affecting 
evidence use; definitions of evidence; 
theories used in included studies 

33 studies examine research uptake 
(amount/rate), 50 examine processes of 
research uptake, 18 examine the impact of 
research use 



Descriptive synthesis; methodological QA Descriptive synthesis; no QA 



Personal contact between researchers and 
PMs; timeliness and relevance of research, 
with clear recommendations and high 
quality; research confirming current policy 

Absence of personal contact between 
researchers and policymakers; lack of 
timeliness or relevance; mutual mistrust 
between scientists and policymakers 
Power and budget struggles 

Weiss, Caplan; two-communities thesis 

Studies partially support common beliefs 
about barriers and facilitators, with little 
empirical evidence; no strong 
recommendations about research and 
policy can be made; limited data support 
two-communities and Weiss's theories 



Available, clear and relevant research 
evidence; relationships, collaboration, and 
contact between researchers and PMs; 
timing, practical managerial support 

lack of clear or relevant research evidence 
and costs; lack of timeliness or opportunity; 
lack of PM research skills or awareness 



Range of theories, reports which papers 
used which theory 

Research into managerial and 
organizational barriers may be more 
useful than individual-level; relational 
approach could be used 



policy making, policy would be better. Finally, we 
make suggestions for a new agenda for EBP research. 

We are not advocating against EBP. On the con- 
trary, we believe that better policy decisions would be 
a desirable outcome, and that evidence ought to play 
a role in those decisions. Rather, our conclusions 
from this critical review of the literature are about 
EBP research, rather than EBP itself. Researchers have 
directed their attention at how to increase the impact 
of their own outputs, rather than on understanding 
the processes behind policy change. The support for 
EBP is not as single-minded or vociferous as it was. 
Reflecting on this historical trend, and reasons behind 
any such shift, we present our novel contribution to 
the literature: an illumination of the discourse around 
EBP by comparing theories, methods, and substantive 
approaches with those from other fields, and on this 
basis propose new directions for EBP research. 



Review: evidence-based policy research 

Below, we describe some of the underlying concepts and 
approaches available to researchers attempting to under- 
stand the relationship between research and policy 
processes. 

Theoretical underpinnings 

Innvser et al. described how the literature at the time 
fell into two camps [34]; those supporting the 'two- 
communities' hypothesis, which explores whether barriers 
to research utilisation are mainly driven by cultural 
or institutional differences between researchers and 
decision-makers, and those drawing on Carol Weiss's 
typology of ways of using evidence [36,37]. Policy and 
academic actors were conceptualised as opposing sets of 
actors, with different priorities, languages, practices, and 
priorities. For proponents of this perspective, 'bridging' 
this gap becomes a priority. It seems likely, however, that 
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insisting on the existence of this 'gap' may polarize 
previously neutral actors; and indeed this debate fails 
to recognise that this may be a UK-specific problem 
(cf. to Dutch studies). 

Models of the research and policy processes are, as 
noted by previous theorists of EBP, rarely made explicit 
in evaluations of applied research. Where implicit, a 
simple 'pipeline' model is usually assumed (i.e., that the 
more research is carried out and the higher the quality, 
the bigger the effect on policy and practice) [38-41]. 

A large new theoretical strand within health policy 
focuses on knowledge brokerage/translation as a frame- 
work for understanding use of evidence [42-46]. This 
model can be seen as an extension of Weiss's 'instru- 
mental use' model, or 'enlightening' and 'strategic use' of 
evidence, describing the influence of research on policy, 
and is linked to ideas about 'coproduction' and 'user in- 
volvement' [47]. 

Weiss and Caplan are still major influences on the 
field of EBP research, but the degree to which these 
contributions are appropriately exploited is debatable. 
The linear (direct use) model is usually perceived to 
be superior, quite contrary to Weiss' argument that 
policymakers thought enlightening (indirect) use could 
offer more. This message is often overlooked by research 
(and researchers) in the field, namely that researchers 
need to reflect on the common view that policymakers are 
interest-oriented and indifferent to evidence. Barriers to 
use of research are equated with barriers to direct use 
of research, while the broader concept of enlightening 
(indirect) use is rarely seen as equally relevant and useful. 

While this is still widespread, theoretical learning from 
other fields is filtering into the debate. Researchers in 
policy studies have long seen the policy process itself as 
a contested arena of negotiation [48,49]. The messy, 
complex, and serendipitous nature of policymaking is 
described by Kingdon [50], Weiss [51], Simon [52], and 
Lindblom [53], amongst others. These scholars have 
contributed ideas such as punctuated equilibrium [54], 
policy 'windows' and 'streams' [50] or 'stages' [48], bounded 
rationality [55], and incrementalism [53]. However, the 
degree to which these models are used in planning and 
conducting empirical research is debateable. As Lomas 
notes, one reason why these may have been resisted by 
EBP researchers is because they offer little help and no 
tools to help the aspiring policy- advisor [56]. 

Of course, many of these are theories about policy, 
rather than analysis tools to assist advisors [57] . Cairney 
argues that policy analysts tend to use concepts, such as 
the policy cycle, which have been rejected by policy 
scholars. We would qualify this statement by restricting the 
rejection of such theories to within science and technology 
studies and policy sciences - these ideas are still common 
currency within health policy amongst other fields [58]. 



The influence of these ideas on the health field is 
perhaps increasing, with applications of ethnographic 
methods [59] and actor-network theory applied to 
EBP [60]. However, the majority of studies still use 
over-simple theoretical models [61]. Asserting the existence 
of and describing and prescribing interventions to close the 
'research-policy gap' is a stance which, we argue, is likely to 
perpetuate and even create gaps between the professions. 

Focus of the research 

The main focus of much theoretical and empirical work 
in EBP has traditionally been, implicitly or explicitly, on 
research evidence uptake, primarily peer-reviewed research 
carried out by university-based academics [30,31]. However, 
a third of studies included in our review examined non- 
research data, for example, public health surveillance data, 
strategic needs assessments and other impact assessments, 
geographic information systems [62], or other non-research 
evidence [63-65]. This suggests that policymakers interpret 
and use 'evidence' in a broad sense, which is usually not 
acknowledged by academic commentators [7,10]. 

The reviews described above included a minority of 
studies which aimed to describe the policy process in 
detail, often using a case-study approach. Frequently, 
these focused on the use of a particular type of evidence 
such as economic evaluations [66] or the role of a 
specific piece of evidence [60,67] . Few attempt a descriptive 
contextualisation or ethnographic understanding of the 
policy process, with exceptions, e.g., [68-71]. Virginia 
Berridge's seminal work on the NHS and comparative 
health policy development uses historical methods to 
understand these processes and develop theory around 
them [72]. Others have taken empirical ethnographic 
approaches to understand, for instance, how health 
decision-makers conceptualise and use evidence [73,74]. 
With these exceptions, very few studies have, as yet, taken 
anthropological or historical approaches to understanding 
the role of evidence or research in policy. 

Research in the area thus focuses primarily on how to 
increase uptake of research, on designing and evaluating 
interventions aiming to increase research use, and on 
identifying barriers and facilitators of research use by 
policymakers [74-76]. This pattern of research on 
evidence use skews the debate by focusing on exceptional 
cases of research use in policy-making, rather than the 
normal discharging of statutory business. As Kogan and 
Henkel noted, attempts to improve use of evidence can 
"fail to note how in those areas of policy where data are 
diffuse, and analyses most likely to be strongly influenced 
by value preferences, problems must be identified 
collaboratively between policy-maker and scientist. It 
failed to acknowledge that policy makers have to work 
hard to identify problems, to specify research that 
might help solve them and to receive and use the results 
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of research" [11]. While a comment evaluating the 
Rothschild experiment, the suggestion that EBP research 
does not reflect the range of knowledge-producing or 
policy-making processes can also be levied at much of the 
academic work subsequent to this important study. 

Focusing on the use of research evidence also allows 
researchers to sidestep the rest of the policy process and 
avoid the context of decision-making more widely. Most 
studies focus on single elements of the policy-making 
process - dissemination of evidence, sources, and types 
of source, knowledge transfer, and priority setting - rather 
than trying to characterise the process as a whole. EBP 
research could draw here on an extensive research 
literature in policy analysis from political science, 
starting with studies of what is called 'the policy cycle' or 
'a stages approach [39,77]. Such studies can analyse who 
is making which decisions, about what, and when; the 
distinction between practice, management, governance, 
and policy is rarely spelt out [78]. Clear definitions of 
policy, use of research, and decision making has been 
encouraged for over 60 years, but remains evasive [79] . 

Methods 

Considerable theoretical work has gone into producing 
taxonomies of factors influencing the utilisation of 
evidence, e.g., [36,80]. However, these rich theoretical 
discourses are not reflected in the bulk of the empirical 
literature, which, despite its breadth with respect to the 
sectors and categories of evidence types studied, finds 
fairly consistently that the main factors affecting use of 
evidence are (a) access to relevant and clear information 
and (b) good relationships between researchers and research 
users. This may be due to the methods used in the 
studies: most use only interviews or surveys to ask 
researchers and policy-makers about their perceptions 
about evidence use; very few use methods such as 
participant observation to observe how evidence is 
actually used in practice, or attempt to find documen- 
tary proof of research use (with exceptions; [76]). 
These lists, while important, cannot on their own lead 
us to an improved understanding of the role of evidence 
in the jigsaw of the policy process. 

Another noticeable feature of the evidence base is the 
emphasis given to researchers' own views of research util- 
isation, for example, a study aiming to look at everyday 
working practice, such as that of Taylor-Robinson et al. 
[81], samples primarily health inequalities/ public health 
academics and practitioners rather than decision- 
makers themselves (such as councillors or executives), even 
though they themselves describe lack of contact between 
academics and policymakers as a barrier to use of evidence. 
The majority of academic studies in EBP research are, 
unsurprisingly, written by and for academics, with 
little involvement of policy-makers as co-authors; indicating 



that policy-makers are not involved in developing or 
carrying out relevant research. 

Underlying assumptions of this research and 
critical reflections 

This overview of the literature provides a starting point for 
a more critical engagement with the empirical literature on 
EBP. We provide a broad-brush characterisation of parts 
of the literature which helps to draw out common 
assumptions across the field as a whole and enable 
critical reflection on them. We focus on three such 
assumptions: 1) that the policy-evidence 'gap' needs 
'bridging'; 2) that policy is usually not based on any 
data, and policy requires research evidence, preferably 
evaluative intervention research; and 3) that greater 
use of evidence in policy-making will produce better 
outcomes at a population level. It is by no means the case 
that these assumptions are universally shared among EBP 
researchers, or that we are the first to identify these issues 
[61,82,83]. Nonetheless, a large proportion of the available 
research still rests on an uncritical acceptance of these 
assumptions. Below, we describe the effect of these 
assumptions, justify our rejection of them, and discuss the 
implications of taking a more critical approach. 

Assumption 1: that a policy-evidence 'gap' exists 

The 'evidence-policy gap' is a widely-acknowledged 
construction in policy-related research, asserting the 
existence of two separate communities with their own 
ecosystems and languages [61,83-85]. Much of the 
'knowledge translation' literature, which attempts to 
take a broader perspective on EBP, fails to question 
the assumption that knowledge and practice are two 
separate practices, and the 'joining' or 'bridging' of 
these (depending on the authors' preferred metaphor) is 
the task of EBP researchers; see, e.g., [20,61,83,84,86]. 
However, as Choi recognizes, evidence - and those 
associated with evidence - is just one voice among many 
[87]. We do not yet know how to make that voice more 
helpful nor more influential. 

Recently, a shift away from this dichotomous debate has 
been made, with concepts such as 'knowledge translation' 
or 'transfer' being replaced by ideas about 'learning', 
'contribution', and co-production [88]. These ideas 
frame the relationship between research and policy as 
a two-way negotiation in which both partners learn 
from the other - pragmatically and politically a step 
towards an equality of prioritisation and experience. 
Certainly, this represents a greater openness to seeing 
a broader range of data types as relevant - including, 
for example, contextual, descriptive data as well as 
evaluations and other forms of research evidence. Too 
often, however, this debate is hijacked by methodologists 
from opposing camps wishing to defend their own 
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method in the face of criticism - whether real or perceived, 
e.g., [89,90]. Without clear definitions of 'evidence' and 
'impact' or 'learning', these studies contribute to negative 
stereotypes on both sides, and perpetuate the gap they aim 
to bridge. 

Assumption 2: that policy is usually not based on 
evidence 

Despite the increased literature in the area, there is a 
surprising lack of evidence about how much evidence 
policymakers use. Studies have largely reported policy- 
makers' perceptions of their usage (e.g., [91]), acknowledge 
that it is impossible to tell how much evidence was used by 
policy participants [92], or rely on self-reported measures 
[93]. EBP researchers have tended to interpret this absence 
of any contradictory evidence as a confirmation of their 
belief that policymakers do not use evidence. This is both 
grossly unfair to policymakers who have been shown to 
draw on a wide range of information sources [39], hugely 
over-simplifies the relationship between evidence and 
policy, and, of course, contradicts the avowed principles of 
EBP researchers; viz, that beliefs ought to be based 
on evidence [76]. 

Implicit in the 'barriers and facilitators' approach is an 
assumption that if these factors were alleviated, research 
uptake would increase. However, this is to miss the key 
point, which is that most research in the area studies the 
use of research evidence by policymakers, not what 
knowledge or information policymakers use. This subtle 
shift in emphasis opens up new avenues of enquiry. 
Other information than research evidence might be more 
relevant and timely, two factors seen as top facilitators for 
policymakers' use of evidence [94]. Policymakers may 
prefer to use local information or intelligence such as 
patient or practice level data, or that held by local councils 
(e.g., datasets of rent, crime, and transport) [95]. It 
seems likely that these sources of information have 
been undervalued by evaluation methodologists, who often 
value trial data above other types. A more naturalistic 
approach using empirical methods to study policymakers 
in vivo, would conceive of evidence as one of many 
influences on a decision. 

Assumption 3: that use of more research evidence 
by policymakers would lead to 'better' policy 

We are not the first to note that "[t]he assumption that 
the use of evidence would improve the outcome of the 
policy process remains relatively untested by any form of 
empirical analysis" [96-98]. The absence of robust evalu- 
ation evidence showing that evidence utilisation actually 
leads to better outcomes is widely admitted. The bulk of 
the intervention evidence in EBP uses only research 
utilisation or uptake as an outcome (or, in some cases, 
merely attitudes and intentions regarding research use). 



Nonetheless, it is still widely claimed that decisions made 
in partnership between "politicians and researchers & lay 
people are more likely to result in positive health outcomes" 
[86] and many researchers continue to advocate for 
increased use of research evidence [99]. 

Such claims, where they are not treated as automatically 
self-evident, are usually supported either by anecdotal cases 
of increased evidence use leading to better outcomes, or by 
studies of the impacts of evidence use on process measures 
such as transparency of decision-making [100]. However, 
the value of process-oriented goals is surely questionable, if 
they cannot be shown to lead to improved health, 
wellbeing, social, or other outcomes for the putative 
beneficiaries of the policy in question. The typologies 
of 'research impact' which have dominated much work in 
this area (e.g., [101]) are of limited value without a more 
open debate about the correct metrics for evaluating EBP, 
based on a realistic view of the currently existing evidence 
base [102]. Moreover, much of the commentary around 
the 'impact agenda' focuses on the aspect of academic 
performance management, without wider examination 
of its connection to policy and knowledge practices 
and theories [16]. 

Even in the absence of robust evaluation data, it is 
clear that many of the existing theoretical rationales for 
how evidence utilisation is supposed to improve outcomes 
are inadequate. If the pipeline model of research use were 
correct, it would be possible to demonstrate the impact of 
research on policy and the value of research would be 
judged on its contribution to policy and its quantifiable 
impact [25]. This model "fails the practitioner because the 
literature on which guidelines are based constitutes an 
unrepresentative sample of the varied circumstances and 
populations in which the intervention might be usable or 
unusable [38]. 

New directions for EBP research 

Above, we describe three assumptions commonly found 
in the EBP literature. We critically discuss the reasons 
we believe these assumptions are flawed, and show how 
the existing research conducted on the basis of these 
assumptions is likely to fail to answer the most pressing 
problems in EBP. Here, we describe how these assumptions 
can form the basis for a new programme of research aiming 
to understand the relationship between science and policy. 
We explore the implications of re-framing future studies in 
a new direction, and suggest more explorative perspectives 
and participative methods. 

Firstly, the assumption that the policy-evidence 'gap' 
needs 'bridging'. Most of the studies identified perpetuate 
the division between the 'two communities' in one way or 
another. Approaching researchers and policy-makers 
separately and asking them for their accounts of evidence 
use may be likely to produce these conflicting accounts; 
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similarly, asking researchers about their perceptions of 
what policy-makers do may not be the most sensitive 
way of exploring policy processes. It would be more 
interesting, and more novel, to approach policymakers 
from an unprejudiced stance, to describe their activities, 
and to identify how they populate policy areas and steer 
policies through [53]. Of course, we are not the first to 
suggest this [56,97] - but these studies are generally the 
exception rather than the rule. 

Secondly, the assumption is often held that policy 
is usually not based on any data, and policy requires 
research evidence, preferably evaluative intervention 
research. By concerning themselves with questions 
such as "how [can] the tension between scientific 
rigour and timely relevance to policy-making be handled" 
[103], EBP researchers often fail to acknowledge their 
lack of knowledge about forms and models of the 
impact and contributions of evidence to policy pro- 
cesses, which can lead to the creation of unhelpful 
straw men. 

Finally, the unspoken corollary to both these assumptions 
is that greater use of evidence in policy-making will 
produce 'better policy and better outcomes at a population 
level. Leaving aside the question of what constitutes 'better' 
in a self-evidently political and therefore value-driven 
terrain, for researchers to convincingly argue for the 
increased use of evidence in policy making, they must be 
able to demonstrate the benefits of doing so. The growth of 
the 'applied research sector claims to address this, but often 
restricts output to vague and untested policy and research 
recommendations, about which there is no evidence of 
effectiveness [99]. If the effects of these policy and research 
recommendations are not evaluated these could be 
misguided at best [104]. 

Therefore, we argue that the following issues are of 
outstanding importance and could form the basis for a 
new agenda of EBP research: 

1. Refocus research on influences on and processes of 
policy rather than how to increase the amount of 
evidence used. Researchers in political and policy 
studies, anthropology and history of policy, and 
science and technology studies have provided a 
wealth of insights and rich empirical data on the 
functioning (or otherwise) of the policy-making 
process [13,53,56,68-70,88,105-108]. Understanding 
the daily lives and activities of policy actors can bring 
fresh insights into how 'evidence' is conceptualised, 
the potential roles it may play, and how it fits with the 
other drivers and triggers which affect policy [95,109]. 
Understanding the roles of exceptional individuals, 
such as policy entrepreneurs, and networks in the 
policy process is also recognised as a key research 
area [3,110,111]. 



Dialogue between these fields has not been very 
extensive or productive so far, largely due to 
scepticism on the part of policy scientists, e.g., [108], 
and a lack of engagement with this body of theory 
and empirical data, with recent exceptions [112,113]. 
To take only one example, policy researchers 
observe that policies rarely have a consistent and 
well-defined goal or aim; rather, "solutions become 
joined to problems" in a provisional and largely 
haphazard way, with the same policy taking on 
different goals at different times or in different 
contexts [50,114]. If this is the case, the question 
of how researchers can evaluate whether or not the 
policy has attained its goal, and use the results of 
this evaluation to inform future policy development, 
is largely moot. 

2. Determine what information and evidence is 
normally used as part of policy processes. As we 
point out above, it is likely that the needs and 
practices of policymakers are rarely the subject of 
rigorous study, and are certainly more complex and 
nuanced than can be captured in surveys. Using 
'research and 'policy' as one-size-fits-all concepts 
underestimates the variety of activities and outputs 
involved in each type of process. A legislative 
manoeuvre is very different from a local tailoring of 
licensing hours; applied health research aiming to 
develop interventions about a specific condition is 
worlds away from contemplative studies of models of 
theories of change. Elucidation of the relationships 

at both ends of the spectra ought to be a research 
priority, in place of the dichotomising of activities 
as 'research or 'policy'. Furthermore, attention to 
context is vital; pressures faced by researchers and 
policymakers in low- and middle-income countries 
may be very different from those in Western settings. 
Finally, as argued above, although similar barriers and 
facilitators are often identified across policy settings 
[35], there is variation of practices and processes 
across policy areas. Analysis of these variations 
would be a whole research agenda in itself. A subtler 
interpretation of context, 'policy ^ and 'research' are 
necessary to understand processes, influences and 
impacts, and indeed to develop any meaningful 
abstractions and generalisations - should these be 
possible, which is by no means certain. A compromise 
between the complexity of policy making and 
development of useful frameworks needs to 
be found. 

3. These questions are likely to require a broader range 
of methodologies than usually applied. Experimental, 
ethnographic, and conceptual studies all need to be 
applied to understand the impact of evidence on 
policy and policy processes more generally. Novel 
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theoretical approaches could be phenomenological, 
psychosocial, or interpretive [115-117]. 

4. Develop conceptual clarity around and metrics to 
evaluate 'impact' of research on policy and 
populations. Without clear methods to understand 
how policy works and how it changes in response to 
information, it will be impossible for researchers to 
know whether they have had an effect on policy. 
Research usage could take many forms, from 
agenda-setting, to provision of policy options, to 
challenging debates, or refuting arguments. Impact 
could therefore be a change in policy, consistency in 
policy, changes in population level outcomes. 
Measures proposed thus far tend to focus on 
citations or mentions of work by policymakers. We 
feel this addresses only a narrow aspect of potential 
impacts, namely awareness of research, which may 
not translate into action. Furthermore, this metric-led 
approach tends to ignore the indirect means by 
which research and evidence of all kinds fit into 
policy - whether by sustaining the status quo, or by 
leading to decisions for change. More attention to the 
variety of impacts and effects leading from research 
may help to develop this debate. 

5. Finally, our analysis of the literature suggests that 
new methods and organisations aiming to bring the 
processes of research and policy closer together are 
likely to further our understanding of the 
relationship between these two types of activities. 
Co-creation and co-production of knowledge are 
lauded as a more democratic, and potentially more 
useful, type of learning activity than many other 
knowledge exchange events [118]. If universities 
were to provide assistance for local policymakers in 
the analysis of existing data, a relationship of mutual 
benefit could start to develop - an end in itself, 
according to reviews of barriers and facilitators of 
evidence use [30,34]. Moreover, such organizations 
would provide natural laboratories for studying the 
role of institutional and organizational factors on the 
practices of policy formation and implementation, 
noted in our review as likely to affect evidence use. 

Perhaps due to the political, financial, and ethical 
pressures on health policymakers to make good decisions, 
health policy leads the way in forming collaborative organi- 
sations to conduct research. Funders in the UK and 
Netherlands have developed specific types of collab- 
orative engagement organisations (e.g., Collaborations 
for Leadership in Applied Health Research and Care, 
CLARHCs, or the health funder ZonMw), which aim to 
bring practitioners and researchers together for mutual 
benefit. There are examples of institutions which spe- 
cifically allow researchers and policymakers to learn 



about each other's priorities and ways of working 
[56,119]. In general, however, the literature above suggests 
that there are insufficient opportunities and incentives to 
form links with policymakers directly. 

Conclusions 

The existing literature on EBP has certainly contributed 
to the desirable outcome of better policy decisions and 
acceptance that evidence ought to play a role in those 
decisions. Now, we believe it is time for researchers to 
reflect on their assumptions, develop new perspectives, 
and use other methods to tackle the problems of EBP. 
There is a common lack of clarity about what researchers 
understand as 'policy', which can encompass decision 
making, project implementation and evaluation, and 
service reconfiguration. There is, in general, little evidence 
about management and organisation, despite these being 
potentially major factors affecting the policy process [120]. 
The assumptions governing the design and outcomes of 
research into policy will probably be significantly different 
in different institutional areas. It is often not clear what 
constitutes 'a decision', nor who is involved in it, or whether 
research evidence is relevant or timely. This muddles and 
prevents any engagement with discussions about what 
constitutes good and bad policy; or indeed how evidence 
ought to be used. 

Rather than attempting to develop a one-size-fits-all 
(pipeline) model, research in this area should revert to 
observational methods with in-depth descriptions of 
practices and their inherited processes and provide an 
empirical basis for theoretical development which can 
inform future activities. Instead of repeating studies of 
perceptions of barriers and facilitators of use of research 
evidence, appropriate methods must be used to answer 
questions about when, why, how, and who finds what 
type of knowledge sound, timely, and relevant at different 
stages of the policy cycle. 

The position of researchers who wish to influence 
policy is untenable unless there is engagement with 
the questions outlined above. We would also argue that 
researchers who advocate for change (for example, via 
policy and practice recommendations) without evaluating 
the (likely) impact of these changes may have a limited 
effect at best. At worst, they create distance between 
policy and research by demonstrating an understanding 
of the context within which they would like their newly- 
generated knowledge to be used. Without understanding 
the complex processes of policy and knowledge mobilisa- 
tion, researchers who make policy and practice recom- 
mendations may simply be ignored. Ultimately, the role of 
researchers is not to judge the 'quality' of policy making 
on the basis of how much of their research is used. This 
stance is both unhelpful and divisive, blinding researchers 
to the important questions raised above regarding the 
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types of information used, by whom, for what purpose, 
and under which circumstances. Rather, our role as 
scientists ought to be to investigate the processes 
surrounding the use of evidence and policy activities 
more widely and to disseminate findings in order to 
help others make informed decisions, of all kinds. 
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